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ABSTRACT 

The  United  States  Military  has  undertaken  a  five  year  experiment  to  better  understand  shared  belief  in 
hierarchical  and  self-organizing  organizations.  Through  the  ELICIT  framework,  data  collected  from  a 
sample  of  thirty-four  cadets  operating  in  the  one  hour  exercise  indicates  that  hierarchical  organizations 
more  effectively  promote  ubiquity  of  correct  shared  beliefs.  Because  of  omitted  variable  bias  in  the  linear 
models  available,  organizational  efficiency  analysis  was  inconclusive.  Self-organizing  networks, 
however,  are  demonstrated  as  likely  more  efficient  than  hierarchical  networks  once  experimental 
modifications  are  made.  The  intent  of  this  paper  is  to  provide  preliminary  analysis  of  the  first  iteration  of 
ELICIT  experiments,  and  to  submit  a  methodology  for  analyzing  organizational  efficiency  and 
effectiveness  under  a  shared  belief  concept. 
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INTRODUCTION 

Much  effort  has  been  put  forth  to  understand  shared  belief  within  an  organization.  Many 
problems  manifest  themselves  in  analysis  of  organizational  shared  belief  These  obstacles  stem  from  both 
the  difficult  nature  of  quantifying  shared  belief  and  measuring  organizational  performance.  By  utilizing 
an  inter-departmental  study  conducted  at  the  United  States  Military  Academy,  this  article  provides 
methodology  to  overcome  both  of  these  obstacles. 

The  experiment,  ELICIT  (Experimental  Eaboratory  for  Investigating  Information-sharing 
Collaboration  and  Trust),  entails  “a  series  of  online  exercises  to  compare  the  relative  efficiency  and 
effectiveness  of  traditional  command  and  control  (C2)  vs.  self-organizing,  peer-based  edge  (E) 
organizational  forms  in  performing  tasks  that  require  decision  making  and  collaboration.”  ^  Through  data 
manipulation  and  panel-data  linear  regression,  it  is  possible  to  understand  what  characteristics  on  the 
individual-  and  organizational-level  contribute  or  detract  from  efficiency  and  effectiveness. 

Regression  analysis  shows  that  hierarchical  organizations  are  better  suited  to  achieving  correct 
self  belief  and  are  therefore  more  effective.  Self-organizing  networks  show  signs  of  high  efficiency,  but 
more  thorough  experimentation  in  larger  numbers  is  needed  to  confirm  this  result.  Not  surprisingly,  the 
most  consistent  factor  in  efficiency  and  effectiveness  is  simply  the  amount  of  time  that  an  organization 
has  to  achieve  shared  belief 

Further  research  is  planned  for  the  next  four  years.  With  the  results  from  this  first  iteration, 
researchers  will  conduct  analysis  that  promises  to  further  isolate  the  contributory  factors  to  organizational 
efficiency  and  effectiveness. 
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BACKGROUND 


Installed  on  client  computers,  the  ELICIT  software  package^  serves  as  the  platform  application 
for  studying  organizational  efficiency  and  effectiveness.  The  four  phase  experiment  entails  an 
introduction,  practice  round,  a  one  hour  exercise,  and  a  wrap  up.  During  both  the  practice  round  and  the 
actual  exercise,  thirty  four  subjects  are  randomly  assigned  to  one  of  two  organizations:  a  typical 
hierarchically  arrayed  organization  (C2)  and  a  control-free,  self-organizing  organization  (E).  These  two 
organizations  operate  independently  for  the  duration  of  the  exercises. 

The  goal  of  the  organization  is  to  identify  a  terrorist  attack  based  on  bits  of  information 
distributed  around  the  organization.  After  ten  minutes  of  the  one  hour  experiment,  all  of  the  correct 
information  has  been  issued  to  the  organization.  Among  the  correct  bits  of  information,  or  factoids,  are 
also  distributed  false  factoids.  Each  entity  receives  four  factoids,  and  they  must  corroborate  within  the 
organization  to  come  up  with  the  correct  arrangement  of  who,  what,  where,  and  when. 

The  C2  group  is  comprised  of  a  squad  leader,  four  team  leaders,  and  twelve  team  members. 
Communications  among  these  entities  are  restricted  to  the  following  graph  in  Figure  (1): 


Figure  (1):  C2  Communications  Hierarchy 


Each  team  is  dedicated  to  identifying  one  key  element  of  the  terrorist  attack:  who,  what,  where, 
and  when.  The  E  group  is  comprised  of  seventeen  entities  with  full  communication  capability  across  the 
organization.  There  are  no  defined  teams,  but  the  goal  remains  the  same:  positively  identify  the  terrorist 
attack. 

All  entities  have  the  ability  to  post  their  information  on  their  organization’s  website.  Within  the  E 
group,  this  website  is  global  to  the  organization.  The  C2  group  has  separate  websites  for  each  echelon 
(four  teams  and  one  squad  site).  The  hierarchy  in  Figure  (1)  describes  where  each  entity  can  post 
information.  Entities  can  also  share  information  with  other  individual  entities.  Once  an  entity  believes  that 
it  knows  any  number  of  correct  factoids,  it  can  report  its  belief  through  the  “identify”  function. 


Table  (1):  Excerpt  from  Parsed  Dataset 


Id 

time 

correct 

who  n 

what  n 

where  n 

when  n 

complete  n 

share 

receive 

sl 

tl 

tm 

cf 

Morgan  1 

245 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

Chris  1 

374 

2 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

Morgan  1 

3592 

3 

14 

10 

0 

12 

0 

44 

85 

0 

0 

0 

1 

^  Parity  Communications  in  collaboration  with  the  Higgins  Trust  Framework  and  the  SocialPhysics  project 
constructed  the  ELICIT  software  package. 


DATA 


The  experiment  was  run  by  cadets  at  the  United  States  Military  Academy^.  Likewise,  all 
participants  were  cadets.  Subsequently,  the  data  from  the  experiments  is  compiled  into  a  multi-sheet 
Excel  workbook.  Each  entity  has  a  log  organized  by  time  with  the  annotated  time  elapsed,  what  kind  of 
action  the  entity  took,  and  information  about  that  action.  Identify  actions  include  text-box  style  input  for 
who,  what,  where,  when,  and  why.  Shared  and  received  factoids  are  logged  according  to  time,  receiver, 
and  sender.  Role  assignment  into  E  group  and  C2  group  (including  position)  are  logged  at  time  zero. 

Manipulating  this  rich  databank  provides  a  challenge.  Through  scripting  techniques,  the  data  was 
parsed  into  a  .csv.  Table  (1)  is  an  excerpt  from  the  ported  data  set.  Each  observation  is  taken  at  time  and 
corresponds  to  the  instant  that  entity  id  reports  its  belief  When  the  entity  reports,  the  number  of  factoids 
that  it  has  correctly  identified  is  reported  in  correct,  which  ranges  from  one  to  four.  The  number  of 
entities  within  the  organization  who  have  shared  belief  with  the  observed  is  reported  in  whoji,  whatji, 
whereji,  and  whenji  for  the  corresponding  factoid.  The  entities  who  share  belief  in  all  factoids  is 
counted  in  complete  n.  Also  measured  is  the  amount  of  factoids  that  the  entity  has  shared  and  received 
{share,  receive)  cumulatively  from  the  time  that  its  belief  is  posted.  The  entities  position  is  a  binary  static 
variable  measured  in  si,  tl,  tm,  and  cf  (squad  leader,  team  leader,  team  member,  or  control  free  entity  from 
E  group).  Table  (2)  reports  summary  statistics  from  the  dataset: 


Table  (2):  Summary  Statistics  from  Parsed  Dataset 


Mean 

Stdev 

Min 

Max 

time 

1906 

888 

245 

3592 

correct 

2.35 

1.1 

0 

4 

who  n 

8.12 

5.1 

0 

14 

what  n 

4.30 

3.5 

0 

12 

where  n 

5.61 

5.5 

0 

15 

when  n 

6.20 

5.2 

0 

15 

completen 

1.57 

2.4 

0 

8 

share 

13.70 

21.6 

0 

140 

receive 

16.59 

17.3 

0 

93 

si 

0.04 

tl 

0.15 

tm 

0.45 

cf 

0.36 

All  variables  contain  164  observations 


EMPIRICAL  FRAMEWORK 

This  article  utilizes  a  panel-data  linear  regression  to  identify  contributory  factors  of  organizational 
efficiency  and  effectiveness.  Linear  regression  allows  an  experimenter  to  partial  out,  or  control  for,  all 
observed  factors  to  find  the  individual  (or  joint)  effects  of  factors  on  a  single  result.  Organizational 
efficiency  will  be  measured  through  shared  belief;  if  many  entities  in  the  network  have  shared  belief,  the 
organization  has  efficiently  distributed  pertinent  information.  Organizational  effectiveness  will  be 
measured  through  correctness;  effective  organizations  are  able  to  achieve  their  goals.  The  goal  of  the 
organizations  in  this  experiment  is  to  correctly  identify  a  terrorist  attack.  First,  consider  the  following 
model  of  organizational  effectiveness  in  Equation  (1)  and  Table  (3): 


^  Under  the  purview  of  Professor  Moxley 


¥  =  Pa  +PxXi  ^PiXi  +PA  +PA  +A6Z‘  +  ^  (1) 


Table  (3)^  Variable  Definitions  for  Equation  ( 1 ) . 


Definition 

w 

Factoids  correct 

X, 

Factoids  received 

Xi 

Factoids  sent 

<t>x 

Squad  leader 

^2 

Team  leader 

^3 

Team  member 

T 

Time  accrued 

S 

Structural  error  term 

There  are  omitted  variables  in  s  that  will  affect  the  estimates  of  the  model:  it  is  likely  that  the 
entities’  ability  will  affect  its  ability  to  positively  identify  factoids.  There  is  no  measurement  error  present, 
since  the  experiment  occurs  over  a  software  platform  which  we  can  expect  to  produce  accurate  data. 
Next,  consider  the  following  model  of  organizational  efficiency  in  Equation  (2)  and  Table  (4): 

^  P\X\+  PlXl+  Pxl>\  +  PJ>2  +  +  PfJ  +  ^  (  1 ) 

Table  (4):  Variable  Definitions  for  Equation  (1). 


Definition 

7] 

Entities  with  shared  belief 

Xx 

Factoids  received 

Xi 

Factoids  sent 

<t>x 

Squad  leader 

^2 

Team  leader 

^3 

Team  member 

T 

Time  accrued 

S 

Structural  error  term 

The  same  omitted  variable  of  entity  ability  will  likely  cause  bias  in  the  estimators.  Future 
experiments  should  record  information  to  pull  it  out  of  the  structural  error  term.  Nonetheless,  the  same 
demographic  was  used  for  the  experiments — cadets  from  the  United  States  Military  Academy.  Since  the 
participants  are  similar,  the  effect  of  the  bias  is  not  likely  detrimental  on  the  estimators. 


RESULTS 


Table  (5)  reports  the  results  of  regression  of  Equation  (1): 


Table  (5):  Results  of  Regression  from  Equation  (1) 

(1)  (2) 

(3) 

share 

.000 

.004 

.006 

(.003) 

(.004) 

(.004) 

receive 

.021 

.011 

-.005 

(.004) 

(.005) 

(.006) 

squad  leader 

1.039 

1.437 

(.342) 

(.359) 

team  leader 

.438 

.538 

(.209) 

(.204) 

team  member 

-.257 

-.442 

(.191) 

(.209) 

.024 

time/60 

(.007) 

Robust  standard  errors  reported  in  parenthesis.  Statistieally  signifieant  estimators  at  95%  eonfidenee  level 

emboldened. 


It  is  clear  from  the  regression  that  as  we  add  more  and  more  explanatory  terms,  the  estimators 
stabilize.  The  most  noticeable  increase  in  statistical  significance  occurs  when  time  is  included  in  the 
regression.  It  is  duly  noted  that  as  time  increases,  the  organization  should  hone  in  on  the  correct  factoids. 

The  findings  of  this  study  pertain  in  a  significant  fashion  to  sharing  of  information.  The  amount 
of  information  that  an  entity  shares  has  no  significant  correlation  with  the  correctness  of  its  belief; 
likewise,  entities  receiving  more  information  have  no  better  chance  of  being  correct.  Squad  Leaders  and 
team  leaders  are  much  more  likely  to  have  correct  beliefs  than  under  the  E  group  (control  free). 
Unfortunately,  team  members  of  the  C2  group  had  significantly  less  accurate  beliefs  than  their  control 
free  counterparts.  Table  (6)  reports  the  results  of  regression  of  Equation  (2): 


Table  (6): 

Results  of  Regression  from  Equation  (1) 

who 

what 

where 

when 

complete 

share 

.0347 

.014 

.018 

-.004 

-.002 

(.009) 

(.009) 

(.018) 

(.02) 

(.009) 

receive 

-.048 

.002 

-.065 

-.052 

-.016 

(.019) 

(.015) 

(.034) 

(.035) 

(.019) 

squad  leader 

.202 

-2.27 

6.96 

2.94 

.512 

(1.24) 

(.659) 

(1.73) 

(1.42) 

(.818) 

team  leader 

-.837 

-1.19 

5.32 

-.513 

-.029 

(.609) 

(.563) 

(.852) 

(1.163) 

(.484) 

team  member 

-3.03 

.218 

2.65 

-1.83 

-.134 

(.679) 

.289 

(.486) 

.153 

(.645) 

.26 

(.857) 

.223 

(.371) 

.092 

time/60 

(.021) 

(.019) 

(.029) 

(.029) 

(.092) 

Robust  standard  errors  reported  in  parenthesis.  Statistically  significant  estimators  at  95%  confidence  level 

emboldened. 


We  can  attribute  differences  to  the  competency  omission  discussed  in  the  empirical  framework 
section.  Each  team  in  the  C2  group  is  responsible  for  a  single  factoid,  so  the  estimates  are  subject  to  bias 
from  team  leader  and  aggregate  team  competency.  Nonetheless,  the  indicators  for  an  exact  shared  belief 
match  of  all  factoids,  measured  in  complete,  show  that  time  has  the  only  consistent  impact  on 
organizational  efficiency.  In  other  words,  the  experiment  shows  that  the  self-organizing  network  was  not 
any  more  efficient  in  terms  of  the  organizational  goal  than  the  hierarchical  network.  On  the  other  hand, 
self-organizing  networks  had  shared-belief  in  who  and  what  factoids  more  than  hierarchical.  The  findings 
indicate  that  future  experiments  need  to  control  for  ability  on  the  individual  level  to  partial  out  the  effects 
of  hierarchical  versus  self-organizing  organizations. 

CONCLUSION 

The  results  on  effectiveness  may  be  surprising  to  some.  Proponents  of  a  self-organizing 
organization  will  be  disappointed  with  the  results  from  this  study,  but  there  is  hope  in  controlling  out 
personal  ability.  Because  no  data  was  stored  linking  the  randomized  entity  assignments  to  social  security 
numbers,  this  study  is  unable  to  control  for  competence  at  the  individual  level.  Likewise,  the  results  from 
studying  efficiency  send  mixed  signals.  Controlling  for  competence  will  likely  produce  more  consistency 
in  the  estimators  of  Table  (5). 

Further  research  into  shared  belief  in  organizations  will  surely  benefit  from  the  methodology 
outlined  in  this  paper.  Efficiency  and  effectiveness  can  be  abstractions;  quantifying  them  in  an 
organization  is  the  principal  order  of  business.  Einear  regression  has  been  illustrated  as  an  effective  tool 
to  partial  out  the  most  influential  characteristics  of  network  behavior — and  elicit  elusive  variations  from 
the  numbers. 
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